Robustness-Driven Resilience Evaluation of Self-Adaptive Software Systems

An increasingly important requirement for certain classes of software-intensive systems is the ability to self-adapt their structure and behavior at run-time when reacting to changes that may occur to the system, its environment, or its goals. A major challenge related to self-adaptive software systems is the ability to provide assurances of their resilience when facing changes. Since in these systems, the components that act as controllers of a target system incorporate highly complex software, there is the need to analyze the impact that controller failures might have on the services delivered by the system. In this paper, we present a novel approach for evaluating the resilience of self-adaptive software systems by applying robustness testing techniques to the controller to uncover failures that can affect system resilience. The approach for evaluating resilience, which is based on probabilistic model checking, quantifies the probability of satisfaction of system properties when the target system is subject to controller failures. The feasibility of the proposed approach is evaluated in the context of an industrial middleware system used to monitor and manage highly populated networks of devices, which was implemented using the Rainbow framework for architecture-based self-adaptation

Testing the robustness of controllers for self-adaptive systems

Self-Adaptive systems are software-intensive systems endowed with the ability to respond to a variety of changes that may occur in their environment, goals, or the system itself, by adapting their structure and behavior at run-time in an autonomous way. Controllers are complex components incorporated in self-adaptive systems, which are crucial to their function since they are in charge of adapting the target system by executing actions through effectors, based on information monitored by probes. However, although controllers are becoming critical in many application domains, so far very little has been done to assess their robustness. In this paper, we propose an approach for evaluating the robustness of controllers for self-adaptive software systems, aiming to identify faults in their design. Our proposal considers the stateful nature of the controller, and identifies a set of robustness tests, which includes the provision of mutated inputs to the interfaces between the controller and the target system (i.e., probes). The feasibility of the approach is evaluated on Rainbow, a framework for architecture-based self-adaptation, and in the context of the Znn.com case study

Advancing Software Services Robustness. Techniques for Assessment and Improvement

Tese de doutoramento em Ciências e Tecnologias da Informaçã

A Black Box Tool for Robustness Testing of REST Services

REST services are nowadays being used to support many businesses, with most major companies exposing their services via REST interfaces (e.g., Google, Amazon, Instagram, and Slack). In this type of scenarios, heterogeneity is prevalent and software is sometimes exposed to unexpected conditions that may activate residual bugs, leading service operations to fail. Such failures may lead to nancial or reputation losses (e.g., information disclosure). Although techniques and tools for assessing robustness have been thoroughly studied and applied to a large diversity of domains, REST services still lack practical approaches that specialize in robustness evaluation. In this paper, we present a tool (named bBOXRT) for performing robustness tests over REST services, solely based on minimal information expressed in their interface descriptions.We used bBOXRT to evaluate an heterogeneous set of 52 REST services that comprise 1,351 operations and t in distinct categories (e.g., public, private, in-house).We were able to disclose several different types of robustness problems, including issues in services with strong reliability requirements and also a few security vulnerabilities. The results show that REST services are being deployed preserving software defects that harm service integration, and also carrying security vulnerabilities that can be exploited by malicious users

Testing data-centric services using poor quality data: from relational to NoSQL document databases

Abstract Businesses are nowadays deploying their services online, reaching out to clients all around the world. Many times deployed as web applications or web services, these business-critical systems typically perform large amounts of database operations; thus, they are dependent on the quality of the data to provide correct service to clients. Research and practice have shown that the quality of the data in an enterprise system gradually decreases overtime, bringing in diverse reliability issues to the applications that are using the data to provide services. These issues range from simple incorrect operations to aborted operations or severe system failures. In this paper, we present an approach to test data-centric services in presence of poor quality data. The approach has been designed to consider relational and NoSQL database nodes used by the system under test and is based on the injection of poor quality data on the database–application interface. The results indicate the effectiveness of the approach in discovering issues, not only at the application-level, but also in the middleware being used, contributing to the development of more reliable services

Applying Text Analytics for Studying Research Trends in Dependability

The dependability of systems and networks has been the target of research for many years now. In the 1970s, what is now known as the top conference on dependability-The IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)-emerged gathering international researchers and sparking the interest of the scientific community. Although it started in niche systems, nowadays dependability is viewed as highly important in most computer systems. The goal of this work is to analyze the research published in the proceedings of well-established dependability conferences (i.e., DSN, International Symposium on Software Reliability Engineering (ISSRE), International Symposium on Reliable Distributed Systems (SRDS), European Dependable Computing Conference (EDCC), Latin-American Symposium on Dependable Computing (LADC), Pacific Rim International Symposium on Dependable Computing (PRDC)), while using Natural Language Processing (NLP) and namely the Latent Dirichlet Allocation (LDA) algorithm to identify active, collapsing, ephemeral, and new lines of research in the dependability field. Results show a strong emphasis on terms, like 'security', despite the general focus of the conferences in dependability and new trends that are related with 'machine learning' and 'blockchain'. We used the PRDC conference as a use case, which showed similarity with the overall set of conferences, although we also found specific terms, like 'cyber-physical', being popular at PRDC and not in the overall dataset

Using Fault Injection to Assess Blockchain Systems in Presence of Faulty Smart Contracts

Authors' manuscript. Published in IEEE Access 2020. The final publication is available at IEEE via http://dx.doi.org/10.1109/ACCESS.2020.3032239Blockchain has become particularly popular due to its promise to support business-critical services in very different domains (e.g., retail, supply chains, healthcare). Blockchain systems rely on complex middleware, like Ethereum or Hyperledger Fabric, that allow running smart contracts, which specify business logic in cooperative applications. The presence of software defects or faults in these contracts has notably been the cause of failures, including severe security problems. In this paper, we use a software implemented fault injection (SWIFI) technique to assess the behavior of permissioned blockchain systems in the presence of faulty smart contracts. We emulate the occurrence of general software faults (e.g., missing variable initialization) and also blockchain-specific software faults (e.g., missing require statement on transaction sender) in smart contracts code to observe the impact on the overall system dependability (i.e., reliability and integrity). We also study the effectiveness of formal verification (i.e., done by solc-verify) and runtime protections (e.g., using the assert statement) mechanisms in detection of injected faults. Results indicate that formal verification as well as additional runtime protections have to complement built-in platform checks to guarantee the proper dependability of blockchain systems and applications. The work presented in this paper allows smart contract developers to become aware of possible faults in smart contracts and to understand the impact of their presence. It also provides valuable information for middleware developers to improve the behavior (e.g., overall fault tolerance) of their systems.This work was supported in part by the Bi-Lateral FCT-NKFIH Program Portugal-Hungary, through the Project Advanced Analytics for Empirical Assessment of Cloud Resilience, in part by the European Union's Horizon 2020 Research and Innovation Program through the Marie Sklodowska-Curie under Grant 823788 ``ADVANCE," the BME-Arti cial Intelligence TKP2020/IK grant of NRDI, in part by the NRDI Fund Based on the Charter of Bolster Issued by the NRDI Of ce under the Auspices of the Ministry for Innovation and Technology, and in part the ÚNKP-19-3 New National Excellence Program of the Ministry for Innovation and Technology